Data Analysis for Business - Final Project

Standard Deviated Group

Group members:

• Edoardo Cocciò - 282401
• Alexandra Tabarani - 282091
• Marta Torella - 284091
• Simone Filosofi - 284531

Task 1

The dataset we are going to analyze is the “Telecom Churn” dataset, which contains information about US Telecom. For this first task, our objective is to predict customer churn, enabling the company to implement preventative measures to retain clients; as a matter of fact, by identifying customers at high risk of churn, the company can proactively offer targeted promotions to improve retention rates.

EDA

We start by performing an exploratory data analysis (EDA) to understand the structure of the dataset and the relationships between the variables. Starting with a general overview of the dataset, we can see that it contains 3333 observations and 20 variables. The variables are a mix of numerical and categorical variable; we proceed by factoring the categorical variables in order to better deal with them later.

##  [1] "State"                  "Account.length"         "Area.code"             
##  [4] "International.plan"     "Voice.mail.plan"        "Number.vmail.messages" 
##  [7] "Total.day.minutes"      "Total.day.calls"        "Total.day.charge"      
## [10] "Total.eve.minutes"      "Total.eve.calls"        "Total.eve.charge"      
## [13] "Total.night.minutes"    "Total.night.calls"      "Total.night.charge"    
## [16] "Total.intl.minutes"     "Total.intl.calls"       "Total.intl.charge"     
## [19] "Customer.service.calls" "Churn"

Missing Values and Duplicates

A crucial step in the data analysis process is to check for missing values and duplicates in the dataset to ensure that the data is clean and ready for analysis; in our case we have no missing values and no duplicates. After ensuring dataset integrity, we proceed with the analysis.

Univariate and Bivariate Analysis

We proceed by conducting an univariate analysis of the variables in our dataset: our first step is to check the distribution of the target variable ‘Churn’; indeed, we identified the variable ‘Churn’ as the target of our analysis since, as previously stated, the goal of our first task is to build a model to predict customer churn. As shown by the below pie chart,the distribution of the target variable is unbalanced, with 85.5% of the customers not churning and 14.5% churning.

We now proceed with the univariate analysis of the numerical variables in the dataset. We will plot the distribution of the numerical variables by the target variable ‘Churn’ to see if there are any differences in the distribution of the numerical variables between the two levels of the target variable.

The above univariate analysis of the numerical variables shows that the variables distribution differs remarkably for the two levels of the target variable “Churn” in a couple of cases. Indeed, there’s a noticeable difference in the distribution of customer service calls: customers who churned seem to make more customer service calls, which could indicate a relationship between customer satisfaction and churn. It also appears that customers who churned tend to have slightly higher total day minutes and total day charges; this might suggest that customers with higher usage during the day are more likely to churn. The other variables do not show a significant difference in distribution between the two levels of the target variable.

Another remarkable imbalance can be found in the distribution of both International.plan and Voice.mail.plan. The majority of customers do not have an international plan nor a voice mail plan; this might suggest that these services are not very popular among customers.

Now we decided to further investigate the relationship between the Customer.service.calls variable and the target variable Churn. The below graph seems to confirm our initial intuition: the churn rate increases as the number of customer service calls increases. The graph illustrates that most customers who do not churn typically make few customer service calls, peaking at 0-1 calls and then sharply decreasing. Conversely, churn rates are initially low and remain steady, even as the number of calls increases, suggesting that those who make more than five calls are less likely to churn.

To further prove our hypothesis, we decided to perform the hypothesis testing to determine if the number of customer service calls is associated with the churn rate:

H0: the average number of customer service calls is the same for customers who churned and those who did not

H1: the average number of customer service calls is different for customers who churned and those who did not

chisq.test(data$Customer.service.calls, data$Churn)

## 
##  Pearson's Chi-squared test
## 
## data:  data$Customer.service.calls and data$Churn
## X-squared = 342.67, df = 9, p-value < 2.2e-16

Since the p-value < 0.05, we reject the null hypothesis; therefore the average number of customer service calls is different for customers who churned and those who did not, which confirms our first assumption.

We now proceed with some more hypothesis testing: we explore the relationship between the Voice.mail.plan variable and the target variable Churn:

H0: the average churn rate is the same for customers who have a voice mail plan and those who do not

H1: the average churn rate is different for customers who have a voice mail plan and those who do not

chisq.test(data$Voice.mail.plan, data$Churn)

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  data$Voice.mail.plan and data$Churn
## X-squared = 34.132, df = 1, p-value = 5.151e-09

Since the p-value = 5.151e-09 < 0.05, we reject the null hypothesis; therefore the average churn rate is different for customers who have a voice mail plan and those who do not. Hence we reject the null hypothesis, confirming that the average churn rate is different for customers who have a voice mail plan compared to those who do not.

Lastly, we investigate the relationship between the International.plan variable and the target variable Churn:

H0: the average churn rate is the same for customers who have a voice mail plan and those who do not

H1: the average churn rate is different for customers who have a voice mail plan and those who do not

chisq.test(data$International.plan, data$Churn)

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  data$International.plan and data$Churn
## X-squared = 222.57, df = 1, p-value < 2.2e-16

The p-value is less than 2.2e-16, indicating a statistically significant association between having an international plan and customer churn. Therefore, we reject the null hypothesis, confirming that the average churn rate is different for customers who have an international plan compared to those who do not.

To conclude, for the the analysis of categorical variables, we also analysed the distribution of the State variable; the below plot shows the distribution of customers across different states in the US.

The above map displaying churn rates across US states reveals significant regional variations, with higher churn rates observed particularly in Western states like California and Arizona, as well as Southern states such as Texas and Florida. These patterns suggest potential market opportunities for businesses to implement targeted customer retention strategies or competitive market entries. Lower churn rates in other states could indicate effective localized strategies or stronger customer loyalty, serving as benchmarks for regions with higher churn rates. By understanding these geographic differences, companies can tailor their approaches to enhance customer satisfaction and retention, adjust their marketing strategies, and set realistic performance benchmarks tailored to regional market dynamics.

Correlation analysis

The correlation matrix shows that there is a perfect correlation between the charge variables and minutes variables so we can drop all the minutes one we drop Total.day.minutes, Total.eve.minutes, Total.night.minutes, Total.intl.minutes.

Lower-dimensional models

Before proceeding with the full model we need to focus on some lower dimensional model in order to investigate some interesting relationships between the variables. We decided to proceed by implementing a logistic regression model to understand the relationship between the target variable “Churn” and the Customer Service Calls variables: indeed, we want to understand if the number of customer service calls influences the churn rate.

service_calls <- data %>%
  filter(Customer.service.calls > 0)

We start by visualizing the proportion of churn by the number of customer service calls:

We can see that the churn rate increases as the number of customer service calls increases. Just by looking at the above plot we get the intuition that customers who make more service calls are more likely to churn.

Hence, we proceed by fitting the logistic regression model to understand the relationship between the number of customer service calls and the churn rate:

set.seed(1)
data$Churn <- as.factor(data$Churn)
model <- glm(Churn ~ Customer.service.calls, family = binomial, data = data)
summary(model)

## 
## Call:
## glm(formula = Churn ~ Customer.service.calls, family = binomial, 
##     data = data)
## 
## Coefficients:
##                        Estimate Std. Error z value Pr(>|z|)    
## (Intercept)            -2.49016    0.08631  -28.85   <2e-16 ***
## Customer.service.calls  0.39617    0.03456   11.46   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 2758.3  on 3332  degrees of freedom
## Residual deviance: 2627.2  on 3331  degrees of freedom
## AIC: 2631.2
## 
## Number of Fisher Scoring iterations: 4

The output shows a logistic regression analysis where the likelihood of customer churn increases significantly with each additional customer service call, as indicated by the positive coefficient of 0.39617 (p < 2e-16). The model fits the data reasonably with an AIC of 2631.2, suggesting that customer service calls are a relevant predictor of churn. Moreover, the intercept’s p-value (<2e-16) suggests that there is a statistically significant effect of having zero customer service calls on the likelihood of churn and that there is a statistically significant association between the number of customer service calls and an increased likelihood of churn.

## List of 4
##  $ axis.title.x:List of 11
##   ..$ family       : NULL
##   ..$ face         : chr "bold"
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi FALSE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.title.y:List of 11
##   ..$ family       : NULL
##   ..$ face         : chr "bold"
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi FALSE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.text.x :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : num 1
##   ..$ vjust        : NULL
##   ..$ angle        : num 45
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi FALSE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ plot.title  :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : num 0.5
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi FALSE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  - attr(*, "class")= chr [1:2] "theme" "gg"
##  - attr(*, "complete")= logi FALSE
##  - attr(*, "validate")= logi TRUE

By looking at the above graph, there is a clear positive trend showing that as the number of customer service calls increases, the probability of churn also rises significantly. For instance, the probability of churn exceeds 50% (indicated by the red dashed line) as the number of calls approaches approximately 5.5; indeed the graph highlights a critical area for operational improvement.

However, we are dealing with a dataset that is highly imbalanced, with 85.5% of the customers not churning and 14.5% churning. This imbalance can lead to biased models that predict the majority class more accurately than the minority class. To address this issue, we will use the undersampling technique to balance the dataset; that is we reduce the number of observations in the majority class (Non-Churn) to match the number of observations in the minority class (Churn).

set.seed(1)
majority_indices <- which(data$Churn == 'False')
minority_count <- sum(data$Churn == 'True')
sampled_indices <- sample(majority_indices, size = minority_count, replace = FALSE)
balanced_data <- data[c(sampled_indices, which(data$Churn == 'True')), ]
balanced_data$Churn <- as.factor(balanced_data$Churn)
balanced_model <- glm(Churn ~ Customer.service.calls, family = binomial, data = balanced_data )
summary(balanced_model)

## 
## Call:
## glm(formula = Churn ~ Customer.service.calls, family = binomial, 
##     data = balanced_data)
## 
## Coefficients:
##                        Estimate Std. Error z value Pr(>|z|)    
## (Intercept)            -0.54195    0.10269  -5.278 1.31e-07 ***
## Customer.service.calls  0.29344    0.04362   6.727 1.73e-11 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 1339.2  on 965  degrees of freedom
## Residual deviance: 1289.9  on 964  degrees of freedom
## AIC: 1293.9
## 
## Number of Fisher Scoring iterations: 4

In this balanced model, the intercept is -0.54195, indicating the log-odds of churn when there are zero customer service calls. Each additional customer service call increases the probability of churn by 0.29344 with a highly significant p-value (1.73e-11), suggesting a strong influence of customer service interactions on churn likelihood. Compared to the initial model, which had a coefficient of 0.39617 for customer service calls, the coefficient in the balanced model is somewhat lower, which might indicate that the initial model overestimated the effect of customer service calls due to the imbalance in the dataset. Furthermore, the AIC (Akaike Information Criterion) has notably decreased from 2631.2 in the original model to 1293.9 in the balanced model, indicating a more parsimonious fit.

The above curve shows a steady increase in the probability of churn as the number of customer service calls rises, beginning from below 50% and reaching towards 90% as the number approaches 7.5 calls; this graph clearly demonstrates how increased customer service interactions are strongly correlated with higher churn rates, emphasizing the need for addressing issues that lead to repeated customer service engagements.

Finally we compare the two confusion matrices of the balanced and unbalanced models:

##                 Predicted Positive Predicted Negative
## Actual Positive                  8                475
## Actual Negative                  5               2845

##                 Predicted Positive Predicted Negative
## Actual Positive                269                214
## Actual Negative                203                280

In the unbalanced dataset, the model struggles to correctly identify positive cases, predicting only 8 as positive (true positives) and incorrectly labeling 5 negatives as positive (false positives). In contrast, the balanced model demonstrates better performance in identifying positive cases (269 true positives) but at the cost of more false positives (203). Indeed, this comparison underscores the impact of using balanced data on model performance, particularly in improving the detection of positive cases, which is critical in scenarios like customer churn prediction.

We further proceeded in exploring the interactions between other variables such as the International.plan and Voice.mail.plan with the target variable Churn.

set.seed(1)
data$Churn <- as.factor(data$Churn)
model <- glm(Churn ~ International.plan*Voice.mail.plan, family = binomial, data = data)
summary(model)

## 
## Call:
## glm(formula = Churn ~ International.plan * Voice.mail.plan, family = binomial, 
##     data = data)
## 
## Coefficients:
##                                          Estimate Std. Error z value Pr(>|z|)
## (Intercept)                               -1.8275     0.0620 -29.477  < 2e-16
## International.planYes                      1.5751     0.1464  10.758  < 2e-16
## Voice.mail.planYes                        -1.0552     0.1669  -6.324 2.55e-10
## International.planYes:Voice.mail.planYes   0.8658     0.3018   2.869  0.00412
##                                             
## (Intercept)                              ***
## International.planYes                    ***
## Voice.mail.planYes                       ***
## International.planYes:Voice.mail.planYes ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 2758.3  on 3332  degrees of freedom
## Residual deviance: 2537.8  on 3329  degrees of freedom
## AIC: 2545.8
## 
## Number of Fisher Scoring iterations: 5

The intercept is very significant with a p-value < 2e-16, indicating a strong effect when both plans are absent. Moreover, both International.planYes and Voice.mail.planYes are highly significant. For the interaction term, while international plans are associated with higher churn, voice mail plans seem to mitigate churn risk. However, customers who have both plans are particularly at risk of churning, possibly due to the higher costs or complexities associated with managing multiple service features.

The last association we investigated was the relationship between the Customer.service.calls and International.plan variables with the target variable Churn.

set.seed(1)
data$Churn <- as.factor(data$Churn)
model <- glm(Churn ~ Customer.service.calls*International.plan, family = binomial, data = data)
summary(model)

## 
## Call:
## glm(formula = Churn ~ Customer.service.calls * International.plan, 
##     family = binomial, data = data)
## 
## Coefficients:
##                                              Estimate Std. Error z value
## (Intercept)                                  -2.99830    0.10623 -28.225
## Customer.service.calls                        0.50057    0.03971  12.605
## International.planYes                         2.48925    0.20001  12.446
## Customer.service.calls:International.planYes -0.36309    0.09364  -3.878
##                                              Pr(>|z|)    
## (Intercept)                                   < 2e-16 ***
## Customer.service.calls                        < 2e-16 ***
## International.planYes                         < 2e-16 ***
## Customer.service.calls:International.planYes 0.000105 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 2758.3  on 3332  degrees of freedom
## Residual deviance: 2423.8  on 3329  degrees of freedom
## AIC: 2431.8
## 
## Number of Fisher Scoring iterations: 5

Both International.planYes and Customer.service.calls are highly significant. The interaction term shows that customers with more customer service interactions generally have higher churn rates. However, the negative interaction indicates that the churn-increasing effect of service calls might be mitigated among customers with an international plan, possibly due to different expectations or experiences.

CONCLUSION UP TO NOW:

• we dropped the minutes variables because they are perfectly correlated with the charge variables;

• we saw that the churn rate is higher for customers that have an international plan;

• we saw that the churn rate is higher for customers that do not have a voice mail plan;

• we saw that the churn rate is higher for customers that have a higher number of customer service calls;

• we saw that the churn rate is higher for customers that have higher total day minutes and total day charge;

• we saw that the churn rate is higher for customers that have higher total eve charge, night charge, and intl charge;

• we saw that State also influences the churn rate;

Preprocessing and feature engineering

We should first deal with the 51 states: we can group them by region (north, south, east, west).

We’ll use the regions column for the first models to avoid having high-cardinality encoded columns: we store the variable in order to use it later with more robust models (tree-based).

state.column <- data$State
data$State <- NULL

Now let’s re-do the chi square:

contingency.table <- table(data$Region, data$Churn)
chisq.test(contingency.table)

## 
##  Pearson's Chi-squared test
## 
## data:  contingency.table
## X-squared = 5.9403, df = 4, p-value = 0.2036

H0: Region and Churn are independent, H1: Region and Churn are dependent. Since p-value > 0.05, we fail to reject the null hypothesis, therefore the region does not influence the churn rate. Let’s see the distribution of churn by region:

We can see that the churn rate is not influenced by the region.

Splitting the data and scaling

set.seed(1)
ids.train <- sample(1:nrow(data), size = 0.75 * nrow(data), replace = FALSE)
data.train <- data[ids.train,]
data.val <- data[-ids.train,] 

We oversample the minority class to balance the dataset, now we have 4260 total observations.

data.train.balanced <- ovun.sample(Churn ~ ., data = data.train, method = "over", N=4260)$data
table(data.train.balanced$Churn)

## 
## False  True 
##  2130  2130

In order to to prevent certain columns from dominating the model, we scale it using the training data’s mean and standard deviation:

cols_to_scale <- c("Account.length", "Area.code", "Number.vmail.messages", "Total.day.calls", "Total.day.charge", "Total.eve.calls", "Total.eve.charge", "Total.night.calls", "Total.night.charge", "Total.intl.calls", "Total.intl.charge", "Customer.service.calls")

train.mean <- apply(data.train.balanced[, cols_to_scale], MARGIN = 2, FUN = mean)
train.sd <- apply(data.train.balanced[, cols_to_scale], MARGIN = 2, FUN = sd)

data.train.balanced[, cols_to_scale] <- scale(data.train.balanced[, cols_to_scale], center = train.mean, scale = train.sd)
data.val[, cols_to_scale] <- scale(data.val[, cols_to_scale], center = train.mean, scale = train.sd)

Baseline Logistic Regression Model

Our starting point is a logistic regression model. We will use this model as a baseline to compare the performance of more complex models:

logistic.baseline <- glm(Churn ~ ., data = data.train.balanced, family = "binomial")

The baseline model shows decent accuracy at 83.33%, however, precision at 57.89% and recall at 42.04% indicate areas for potential improvement. This is normal since we still have to perform feature selection and regularization to improve the model.

##              
## baseline.pred False True
##             0   629   48
##             1    91   66

##    Accuracy Precision    Recall        F1
## 1 0.8333333 0.5789474 0.4203822 0.4870849

AIC and BIC Model Selection

We performed the Akaike Information Criterion (AIC) stepwise selection to identify the most important predictors in the model:

akaike.fw <- step(glm(Churn ~ 1, family = "binomial", data = data.train.balanced), scope = formula(logistic.baseline), direction = "forward")
akaike.back <- step(logistic.baseline, direction = "backward")
akaike.both <- step(logistic.baseline, direction = "both")

Across all selection methods, certain predictors consistently appear as significant: “International.plan”, “Customer.service.calls”, “Total.day.charge” and “Voice.mail.plan”.

Bayesian Information criterion (BIC) is another criterion to select the best model. We will use the BIC to compare the models selected by the AIC:

bayesian.fw <- step(glm(Churn ~ 1, family = "binomial", data = data.train.balanced), scope = formula(logistic.baseline), direction = "forward", k = log(nrow(data.train.balanced)))
bayesian.back <- step(logistic.baseline, direction = "backward", k = log(nrow(data.train.balanced)))
bayesian.both <- step(logistic.baseline, direction = "both", k = log(nrow(data.train.balanced)))

Considering bidirectional elimination of both methods, BIC is more strict as it removes all the covariates representing the regions.

After that we compare the validation set results:

The results show that the model selected by the AIC method performs slightly better than the one selected by the BIC method. The AIC model has a higher accuracy, precision, recall, and F1 score: maybe dropping all the region features as BIC does is too big of a loss of informations.

AIC:

##             
## akaike.preds False True
##            0   631   47
##            1    89   67

##    Accuracy Precision    Recall        F1
## 1 0.8369305 0.5877193 0.4294872 0.4962963

BIC:

##               
## bayesian.preds False True
##              0   627   48
##              1    93   66

##    Accuracy Precision    Recall        F1
## 1 0.8309353 0.5789474 0.4150943 0.4835165

ROC Curves

The AUC-ROC curve is a graphical representation of the model’s ability to discriminate between positive and negative classes. We plot the ROC curves for the full model, the AIC model, and the BIC model to compare their performance:

• The full model has an AUC of 0.726 and it’s clearly the model with the lowest discrimination ability;

• The AIC model (AUC of 0.825) and the BIC model (AUC of 0.834) have two very similar AUCs, but the BIC model has a slightly better performance.

AUC: In order to insight into the models’ ability to evaluate their predictive performance in discerning churn, we’re calculating the Area Under the Curve (AUC) for Akaike, Bayesian, and Baseline models.

auc_akaike <- as.numeric(auc(roc(data.val$Churn, predict(akaike.both, newdata = data.val, type = "response"))))
auc_bayesian <- as.numeric(auc(roc(data.val$Churn, predict(bayesian.both, newdata = data.val, type = "response"))))
auc_full <- as.numeric(auc(roc(data.val$Churn, baseline.pred)))

LASSO and RIDGE REGULARIZATION

We leverage both LASSO and Ridge regularization methods to fine-tune our logistic regression model, aiming to enhance its predictive capabilities while addressing potential multicollinearity and feature redundancy.

LASSO:

After tuning over a grid of parameters, the highest achieved accuracy is 77.42%. The optimal regularization parameters selected are alpha = 1 and lambda = 0, indicating LASSO regularization without penalty.

## [1] 0.7741784

##   alpha lambda
## 1     1      0

RIDGE:

Using the Ridge regularization technique with the glmnet method and cross-validation, we achieved a maximum accuracy of 77.07%. The best-performing parameters selected are alpha = 0 and lambda = 0.01034, indicating Ridge regularization without penalty.

## [1] 0.7706573

##   alpha     lambda
## 3     0 0.01034483

As shown in the above plots, the AUC for the Lasso model is 0.778, while the AUC for the Ridge model is 0.776.

Now let’s compare the metrics obtained with the different models:

##            Model  Accuracy Precision    Recall  F1_Score       AUC
## 1         Akaike 0.8369305 0.5877193 0.4294872 0.4962963 0.8249269
## 2       Bayesian 0.8309353 0.5789474 0.4150943 0.4835165 0.8343567
## 3  Full Logistic 0.8333333 0.5789474 0.4203822 0.4870849 0.7262792
## 4 Logistic Ridge 0.7853717 0.7631579 0.3640167 0.4929178 0.7760234
## 5 Logistic Lasso 0.7889688 0.7631579 0.3686441 0.4971429 0.7781067

Our baseline logistic regression model demonstrates moderate performance across various metrics:

• following analyses under different approaches, accuracy ranges from 78.54% to 83.69%;

• precision consistently hovers around 76%, indicating a high percentage of correctly predicted positive cases;

• recall is lower, ranging from 36% to 43%, suggesting room for improvement in capturing all positive cases;

• the F1 score, a balanced metric, varies from 0.49 to 0.50. AUC values, indicating model discrimination, range from 0.73 to 0.83;

Overall, the model performs reasonably well but could benefit from increased recall to capture more positive cases.

Other models

From now on we are studying other models and comparing the metrics with the logistic regression we chose as baseline model:

• Decision trees

• Random Forest

• Gradient Boosting

Decision Trees

First of all we split again to undo the scaling and to make trees more interpretable for us:

set.seed(1)
ids.train <- sample(1:nrow(data), size = 0.75 * nrow(data), replace = FALSE)
data.train <- data[ids.train,]
data.val <- data[-ids.train,] 

Then we oversampled the minority class in order to have a total of 4260 observations as in the previous case:

data.train.balanced <- ovun.sample(Churn ~ ., data = data.train, method = "over", N=4260)$data
table(data.train.balanced$Churn)

Computing the confusion matrix we can notice that the model accurately predicts 651 ‘False’ instances and 100 ‘True’ instances, but misclassifies 14 ‘False’ instances and 69 ‘True’ instances. It achieves an accuracy of 90.05%, with a precision of 87.72% and recall of 59.17%.

##          
## tree.pred False True
##     False   651   14
##     True     69  100

##    Accuracy Precision   Recall        F1
## 1 0.9004796  0.877193 0.591716 0.7067138

Pruning the tree

In order to improve the performance and generalization ability of decision trees (to prevent overfitting) we adopted a Pruning approach with cross-validation:

After pruning, the resulting tree is the following (there are some small differences with respect to the previous tree):

The predictive performance of the pruned tree is evaluated on the validation dataset, giving as contingency table the below:

##            
## pruned.pred False True
##       False   651   14
##       True     69  100

These values inside the contingency table show that the pruned tree model are exactly the same as the full tree model, even if the structure of the tree changed, which is a good sign of the model’s robustness.

Random Forest

Now its the turn of the Random Forest model. Using again a seed set for reproducibility, we train the model on a balanced training dataset, configuring parameters such as the number of trees and variable selection for each split (500).

set.seed(1)
fit.bag <- randomForest(Churn ~ ., data = data.train.balanced, mtry = ncol(data.train.balanced) - 1, ntree = 500, importance = T) 

In this section, we evaluate our bagged model’s performance on the validation dataset. Our aim with this plot is to illustrate the importance of each predictor variable in the random forest model. At the top of our results we find ‘Customer service calls’, ‘International plan’, ‘Total international calls’.

## [1] 0.01296275

## [1] 0.03357314

##                 
## validation.preds False True
##            False   713   21
##            True      7   93

We utilized two metrics for error estimation: Out-of-Bag (OOB) test error, which is approximately 1.33%, and true test error, which is approximately 3.72%. By comparing these figures, we observe that with a larger number of trees, the OOB test error estimation could potentially approach the true test error more closely. Additionally, we examine the contingency table of predicted versus actual outcomes, which illustrates the model’s performance. For instance, out of 720 instances predicted as ‘False’, 709 are actually ‘False’, while out of 114 instances predicted as ‘True’, 94 are indeed ‘True’.

eXtreme Gradient Boosting

We’re now using XGBoost to build a churn prediction model. First, we’ll prepare our data by encoding categorical variables and splitting it into training and validation sets, then, we’ll train the XGBoost model with specified parameters.

X.tr <- model.matrix(Churn ~ ., data = data.train.balanced)[, -1]
y.tr <- as.numeric(data.train.balanced$Churn)-1
X.val <- model.matrix(Churn ~ ., data = data.val)[, -1]
y.val <- as.numeric(data.val$Churn)-1
fit.xg <- xgboost(as.matrix(X.tr), label = y.tr, nrounds = 50, objective = "binary:logistic", eval_metric = "error")

As shown from the results below, the XGBoost model achieves strong performance with a 97% of accuracy, a recall of 93.20% for ‘Churn’ instances and a precision of 84.21%. It exhibits low rates of false negatives (18 out of 114) and false positives (7 out of 731).

##        
## xg.pred False True
##       0   713   18
##       1     7   96

##   Accuracy Precision    Recall        F1
## 1 0.970024 0.8421053 0.9320388 0.8847926

Validation error and number of trees

## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.

The vertical line highlights the number of trees that minimizes the validation error, helping identify the optimal number of trees for the model. In our specific case the value is 35.

Feature Importance

The feature importance plot below illustrates the relative importance of each feature in the XGBoost mode:

Notably, total day minutes, customer service calls, and the presence of an international plan emerge as the top features, indicating their significant influence on customer churn. Other factors such as evening and international call durations also contribute to the model’s predictive power, albeit to a lesser extent.

## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [08:07:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.

Here we can see from the confusion matrix that the model has a good performance, but if compared to the matrix shown at the beginning of the section, we can see that the model improved in the prediction of true positives (96 before, 98 now), but has a slightly worse performance in predicting true negatives (713 before, 707 now).

##           
## pred_xg_cv False True
##      False   707   16
##      True     13   98

Main conclusions

Main conclusions

To sum up, we have performed a comprehensive analysis of the churn dataset, including data exploration, preprocessing, and feature engineering. We have built a logistic regression model as a baseline and compared its performance with other models, such as decision trees, random forests, and gradient boosting.

##               Model  Accuracy Precision    Recall  F1_Score       AUC
## 1            Akaike 0.8369305 0.5877193 0.4294872 0.4962963 0.8249269
## 2          Bayesian 0.8309353 0.5789474 0.4150943 0.4835165 0.8343567
## 3     Full Logistic 0.8333333 0.5789474 0.4203822 0.4870849 0.7262792
## 4    Logistic Ridge 0.7853717 0.7631579 0.3640167 0.4929178 0.7760234
## 5    Logistic Lasso 0.7889688 0.7631579 0.3686441 0.4971429 0.7781067
## 6 CV Decision Trees 0.9004796 0.8771930 0.5917160 0.7067138 0.8906798
## 7    Random Forests 0.9664269 0.8157895 0.9300000 0.8691589 0.9030336
## 8            CV XGB 0.9652278 0.8596491 0.8828829 0.8711111 0.9207968

Now to draw the main conclusions from our analysis let’s compare the metrics obtained with the different models’ final data:

• Best classification model: Random Forests demonstrate the best overall performance, with an accuracy of 96.64%, precision of 81.58%, recall of 93.00%, F1-score of 86.92%, and area under the curve (AUC) of 90.30%. This indicates that Random Forests is the most effective model in predicting churn.

• Boosting models: CV XGBoost shows an accuracy of 96.52%, precision of 85.96%, recall of 88.29%, F1-score of 87.11%, and AUC of 92.08%, demonstrating performance similar to Random Forests and superior to other classification models.

• Linear models: Linear models (Akaike, Bayesian, Full Logistic, Logistic Ridge, Logistic Lasso) exhibit lower performance compared to boosting models and decision trees, with AUC ranging from 72.63% to 83.44%. However, Logistic Ridge and Logistic Lasso show notably higher precision compared to other linear models.

• Decision trees: CV Decision Trees demonstrate good performance with an accuracy of 90.05%, precision of 87.72%, recall of 59.17%, F1-score of 70.67%, and AUC of 89.07%, but are surpassed by boosting models and Random Forests in terms of AUC.

In conclusion, Random Forests and CV XGBoost emerge as the most effective models for churn prediction, with Random Forests showing slightly better performance in terms of AUC.

Task 2

For this second task, we will focus on the customer segmentation. We will use clustering techniques (kmeans and hierarchical clustering) to group customers based on their characteristics and behaviour. The goal is to identify different customer segments that can help the company to better understand its customers and tailor its marketing strategies.

Feature Engineering

Before proceeding with the clustering, we need to preprocess the data and engineer some features. Indeed, we drop the columns that are not useful for the clustering analysis, we create three new features that can help to better identify the customer segments, and lastly, we scale the numerical variables.

##   Number.vmail.messages Total.minutes Total.charge Total.calls
## 1             1.2346975     1.3933218   1.53397886 -0.06203854
## 2             1.3077522     0.3705797  -0.01997227  0.77980605
## 3            -0.5916711    -0.5832384   0.27044139  0.80883517
## 4            -0.5916711    -0.3008727   0.69987274 -1.45543647
## 5            -0.5916711    -0.8878377  -0.70077807  1.56359239
## 6            -0.5916711     0.6929658   0.77699899  0.51854394

As shown by the above table, we decided to consider the following features for the clustering analysis: Number.vmail.messages, Total.minutes, Total.charge, Total.calls. These features were chosen to capture a comprehensive view of customer behaviors, focusing on voicemail usage, overall call time, total expenditure, and call frequency. This selection aims to segment the customer base effectively, revealing patterns that can inform targeted marketing strategies, service optimizations, and customer retention efforts. By analyzing both behavioral and financial dimensions, the company can better understand diverse customer needs and tailor its approaches to enhance satisfaction and loyalty.

Kmeans

Once we have preprocessed the data, we can proceed with the clustering; the first method we will use is the kmeans algorithm. We start by determining the optimal number of clusters using the elbow method and the silhouette score. As shown in the plots below, both the elbow method and the silhouette score suggest that the optimal number of clusters is 3; we will then proceed with 3 clusters, as suggested by both the methods.

After feeding the data into the kmeans algorithm, we can visualize the clusters thanks to the Principal Component Analysis (PCA) that helps us in reducing the dimensionality of the data and visualizing the clusters in a 2D space. Hence, the plot below shows the customer segments identified by the kmeans algorithm.

K <- 3
set.seed(1)  
km_result <- kmeans(clustering_data, centers = K, nstart = 25, iter.max = 100)

The above scatter plot shows the customer segments identified by the kmeans algorithm. Each color represents a different cluster, and the points are plotted in a 2D space using the first two principal components. While the blue and the red clusters are well separated, indicating that the kmeans algorithm has successfully grouped customers based on their characteristics and behavior, the clusters exhibit considerable overlap, especially between the red and green clusters, indicating some shared characteristics among them.

## [1] 0.3016074

##   cluster size ave.sil.width
## 1       1 1249          0.32
## 2       2 1318          0.32
## 3       3  766          0.25

The silhouette plot for three clusters reveals varied cluster cohesion: Cluster 2 (Green) displays high silhouette widths, indicating strong cluster integrity and good separation from other clusters. In contrast, Clusters 1 (Red) and 3 (Blue) show lower and more varied silhouette widths, suggesting these clusters might be less well-defined and contain points that are close to the boundaries of other clusters. The average silhouette width across all clusters is 0.3, highlighting an overall moderate separation quality between the clusters. This suggests potential areas for improving the clustering approach or further investigation into the data characteristics.

Hierarchical Clustering

The second clustering method we will use is hierarchical clustering; we decided to use the same variables as for the kmeans algorithm in order to compare the results obtained with the two methods. We use the correlation-based method for hierarchical clustering which uses correlation coefficients as a measure of similarity between observations. We also consider different linkage methods to build the dendrogram and identify the optimal number of clusters: complete linkage, single linkage, average linkage, centroid linkage, and ward linkage.

dist_matrix <- dist(clustering_data)
set.seed(1)
h_complete <- hclust(dist_matrix, method = "complete")
h_single <- hclust(dist_matrix, method = "single")
h_average <- hclust(dist_matrix, method = "average")
h_centroid <- hclust(dist_matrix, method = "centroid")
h_ward <- hclust(dist_matrix, method = "ward.D2")

By plotting the dendorgrams for the different linkage methods, we can identify the optimal number of clusters. The Complete Linkage shows a more balanced structure, suggesting uniformity in cluster compactness. Single Linkage results in elongated clusters, prone to chaining effects, highlighting susceptibility to noise. Average Linkage offers a moderate clustering structure, more natural than Single or Complete Linkage. Centroid Linkage shows variations in cluster sizes, influenced by centroid positions which may shift due to outliers. Ward’s Method demonstrates very compact and well-defined clusters, effectively minimizing within-cluster variance, suggesting it as the most robust method for distinct and cohesive grouping in this dataset. As shown in the plots below, the ward linkage method in particular suggests that the optimal number of clusters is 3, which is consistent with the results obtained with the kmeans algorithm.

After identifying the optimal number of clusters, we can proceed by also checking the silouhette values achieved by the different linkage methods. When analyzed alongside dendrograms, high silhouette scores confirm the visual separation and compactness seen in methods like Ward’s, suggesting effective clustering. Conversely, low scores, particularly with methods like Single Linkage which shows elongated clusters, indicate issues such as chaining effects or potential misassignments. Moderate scores for Average or Complete Linkage reflect better clustering than Single but possibly with some overlaps.

##  complete    single   average  centroid   ward.D2 
## 0.1637051 0.4198898 0.2402278 0.3602106 0.2612211

Finally, we proceed with the hierarchical clustering using the ward linkage method and 3 clusters, as suggested by the silhouette scores. We then visualize the clusters using PCA, as we did for the kmeans algorithm. The PCA-based cluster visualization using Ward’s method clearly identifies three distinct customer groups in the dataset: a densely packed central cluster (blue), a broadly distributed cluster with some overlap (red), and a smaller, more distinct cluster (green). These clusters, represented on two principal components that likely capture the most significant variances in customer behavior such as usage, suggest specific strategies for targeted marketing, customer service enhancements, and product development. The graph validates the effectiveness of Ward’s method in hierarchical clustering by showing well-defined group separations, providing robust insights for strategic business decisions focused on tailored customer engagement and service offerings.

The comparison between the PCA plots of k-means clustering and hierarchical clustering using Ward’s method reveals distinct differences in clustering outcomes. K-means clustering exhibits greater overlap and less defined separation between clusters, particularly between the blue and red clusters, suggesting a less effective partitioning of the dataset. In contrast, hierarchical clustering demonstrates clearer and more distinct cluster boundaries, with notably more compact and homogeneous clusters. This indicates that hierarchical clustering with Ward’s method provides more precise and well-defined groupings, enhancing the interpretability and utility of the clustering results. Consequently, hierarchical clustering appears more suitable for this dataset, offering robust insights for targeted strategies and data-driven decisions.